10 - 708 : Probabilistic Graphical Models 10 - 708 , Spring 2014 13 : Variational Inference : Loopy Belief Propagation

The problem of probabilistic inference concerns answering queries about conditional and marginal probabilities in graphical models. Consider two disjoint subsets E and F of the nodes in a graphical model G. A query regarding marginal distribution p(xF ) can be calculated by the marginalization operation ∑ G\F p(x). A query regarding conditional distribution p(xF |xE) can be calculated by p(xF |xE) = p(xF ,xE) p(xE) . A query could also ask to compute a mode of the density x̂ = arg maxx∈Xm p(x). In the previous lectures, we have learnt many exact inference techniques such as naive brute force marginalization, variable elimination and family of message passing algorithms such as sum-product, belief propagation and junction tree. In brute force and variable elimination techniques, individual queries are computed independently and as a result several intermediate terms may be computed repeatedly, while the message passing algorithms allows sharing of intermediate terms and hence is more effective in the long run.